Papers with long document understanding
KALM: Knowledge-Aware Integration of Local, Document, and Global Contexts for Long Document Understanding (2023.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to infuse knowledge graphs with pre-trained LMs are limited by the input sequence length. |
| Approach: | They propose a language model that leverages knowledge in local, document-level, and global contexts for long document understanding. |
| Outcome: | The proposed model achieves state-of-the-art on three long document understanding tasks across 6 datasets/settings. |
ChuLo: Chunk-Level Key Information Representation for Long Document Understanding (2025.findings-acl)
Copied to clipboard
| Challenge: | Traditional approaches to truncate inputs, sparse self-attention, and chunking often lead to information loss and hinder the model’s ability to capture long-range dependencies. |
| Approach: | They propose a novel chunk representation method that uses unsupervised keyphrase extraction to group input tokens to retain core document content while reducing input length. |
| Outcome: | The proposed method minimizes information loss and improves the efficiency of Transformer-based models. |